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Abstract 

Over the past three years, we have presented several papers on a model of 
data and information flow through a system: the Dynamic Model of 
Situated Cognition (DMSC). The DMSC has proved useful in a variety of 
settings: modeling individual performance, military C2, naval operations, 
human error in military mishaps, team behaviors in complex organizations 
and, most recently serving as an aid to system designers. Although first 
proposed as a conceptual model, the DMSC can also be used to assess the 
flow of data and information in a dynamic field setting, the Tactical 
Network Topology (TNT) Project. The TNT project is a series of ongoing 
war-gaming field studies conducted quarterly by the Naval Postgraduate 
School and held at a variety of operational venues. The current research 
involved 12 trials in which four mock enemy vehicles attempted to 
infiltrate a specified region. Unmanned aerial vehicles (UAVs) were used 
as the primary sensor platform for the simulated Joint Force. GPS data for 
enemy vehicles and friendly UAVs, and audio and video tapes of the 
tactical operations center (TOC) were recorded and used to populate the 
DMSC. This study validates the utility of the model, extending its use to 
field settings. 
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Introduction 


The concept of network Centric Warfare (NCW) has been in existence for more than ten 
years (Cebrowski & Garstka, 1998). Over the last decade, NCW proponents have 
challenged both U.S. and allied defense organizations to revolutionize their concepts and 
practices of military command and control. NCW tenets have been developed and 
promulgated. Books have been published. Numerous symposia have been conducted. 
Hundreds of researchers have been funded. Much of the effort put forth by NCW 
enthusiasts has been focused on technology. Improvements in bandwidth, connectivity, 
and processing speed have moved us closer to the time where military practitioners of all 
services, in every part of the battlespace, will have access to the same data. While such 
capability fulfills a portion of the tenets of NCW, it does not necessarily follow that these 
technological improvements will lead to achievement of the other tenets. 

The tenets of Network Centric Warfare are as follows (Alberts, 1996). 

1. A robustly networked force improves information sharing. 

2. Information sharing and collaboration enhance the quality of information and 

shared situational awareness. 

3. Shared situational awareness enables self-synchronization. 

4. These, in turn, dramatically increase mission effectiveness. 

At least some of the phrases within the NCW tenets suggest technological solutions: a 
robustly networked force; information sharing; shared situational awareness; and self¬ 
synchronization. It is reasonable for organizations to justify their research and 
development of technological systems based on these tenets. However, while technology 
can be used to aid the activities listed in the tenets, many of these activities are 
fundamentally human endeavors. Information sharing among military organizations 
facilitates collaboration and is essential to warfighters having shared situation awareness. 
And, shared situation awareness leads to the ability of warfighters to synchronize their 
activities. 

Focusing on technological solutions with little consideration for the capabilities and 
limitations of the warfighters is imprudent. Yet, this is often the case. The result is that 
these novel technologies provide capabilities not needed by warfighters or they function 
in ways that are not compatible with warfighters. These solutions will quickly fall into 
disuse or will distract warfighters from the tasks that are truly important for mission 
accomplishment. Equally unacceptable is the narrow focus of some psychologists on 
cognitive processes of the humans without considering the military context or the 
technologies with which the warfighters must interact. While these researchers may gain 
valuable insights into human cognition, their findings by themselves may have limited 
applicability to warfighting. These findings must be shared with technologists who can 
then incorporate them into the design of new systems. Thus, cooperation and 
collaboration between technologists and psychologists or human performance experts are 
critical to the success of NCW. 
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The Dynamic Model of Situated Cognition (DMSC) has successfully integrated 
technological and human elements into a single conceptual framework (Miller & 
Shattuck, 2004a; 2006). This model describes the interaction between technological and 
human agents in complex systems. It has been used retrospectively to explain how 
activities can go awry in complex systems (Shattuck & Miller, 2006). Tracing the flow 
of events through the model often reveals that, whereas the initial finding may be ‘human 
error,’ the actual cause may be rooted in the technology. The model also has been used 
as an analytical framework for military command and control (C2) simulation 
experiments (Miller & Shattuck, 2004b). The present paper extends the use of the model 
to field experimentation. The authors discuss the challenges and the benefits of using the 
model in dynamic settings. 

The Dynamic Model of Situated Cognition 

The model grew out of a command, control, communications, computers, intelligence, 
surveillance, and reconnaissance (C4ISR) war game conducted at FT Knox in the Army’s 
Unit of Action Mounted Battle Laboratory (UAMBL). In that exercise it was apparent 
that the research analysts were divided into two camps: those focused on the 
technological aspects of the simulation and those focused on the human participants in 
the system. Although the two groups used the same terms (i.e., situation awareness), it 
became apparent that they defined the term in very different ways. The model, then, 
emerged as an attempt to define a common framework with which the groups could 
communicate effectively. 

The Dynamic Model of Situated Cognition has been described elsewhere in detail (Miller 
& Shattuck, 2004a; 2004b; Miller & Shattuck, 2005; Shattuck & Miller, 2006). A brief 
description of the model is presented here. The model consists of six ovals of varying 
sizes and three lenses (see Figure 1). The three ovals on the left side of the model (1, 2 
and 3) represent the technological side of a system while the three ovals on the right (4, 5 
and 6) represent the human perceptual and cognitive processes. Oval 1 is ground truth 
and in the simulated laboratory scenarios where it was first used, contains all the data 
concerning enemy and friendly forces, terrain features, weather conditions, non- 
combatants, and even the intentions of those human entities in the battlespace. 

Oval 2 is a subset of Oval 1, and includes only those entities that are detected by sensor 
systems. It does not include everything in Oval 1 because there may not be enough 
sensors to cover the battlespace, they may be in the wrong place, they may be in the 
wrong mode (i.e., 1 meter vs. 3 meter resolution), or they may be inoperative. Oval 2 is 
the first point at which error may be introduced into the model. A flawed sensor 
algorithm may cause an entity to be misidentified. For example, in Figure 1, a red enemy 
entity in Oval 1 is detected and classified as a green neutral element by the sensor 
network and that misidentification is propagated through the remainder of the model. 
Alternatively, the pink entity in Oval 1 is an enemy decoy but the sensor is unable to 
discriminate between a decoy and an actual enemy vehicle. The technological system 
classifies it as an enemy entity and propagates that representation through the model. 
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Figure 1. Dynamic Model of Situated Cognition 


Oval 3 depicts the data displayed at the command and control workstation of the 
individual operator. These displays may be visual, auditory, or tactile. Many displays 
can be tailored by the users and, as such, the type and amount of data displayed may vary 
greatly over time. Ovals 4, 5 and 6 on the right side of the model represent, respectively, 
the perception of data elements, the comprehension of the current situation (sometimes 
called a mental model) and the individual’s projection of current events into the future. 
These three ovals correspond to situational awareness Levels 1, 2, and 3 in the scientific 
literature (Endsley, 2000). 

The lenses in the Dynamic Model of Situated Cognition reflect the knowledge, attributes, 
and attitudes that reside in the decision maker. Although the elements contained in each 
of the three lenses is the same, the placement of the lenses in the model indicates that 
different functions are performed by each lens. Lens A, the lens between Ovals 3 and 4, 
directs attention to selected incoming stimuli. Lens B, between Ovals 4 and 5, influences 
how data are organized into information. Lens C, between Ovals 5 and 6, guides the 
process of extrapolating current information into predictions about the future. In 
addition, the lenses are highly dynamic, and vary from person to person, continually 
being influenced by incoming information, new experiences, and variations in the 
physiological and psychological states of the individual. 
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There are at least six classes of information embedded in the lenses. Individual states 
and traits represent those relatively enduring (e.g., intelligence or personality) and 
transient (e.g., fatigue) characteristics of an individual that affect decision making. Social 
factors include issues ranging from small group dynamics to cultural differences that 
might exist among decision makers. The local context influences the data to which a 
decision maker will attend. The plan represents the specific goals to be achieved and the 
means by which they will be achieved. Guidelines represent general procedures to which 
decision makers may refer if the plan is underspecified. Experience refers to previous 
activities in which a decision maker has engaged. 


As is the case with the human visual lens, perceptual distortions may result from 
asymmetries (see Figure 2). Distortions in the lens preceding the perceptual oval (Lens 
A) may divert the attention of the decision maker away from the most important or 
relevant data. Distortion in the other two lenses (Lens B and C) can result in an 
inaccurate mental model of the current situation and false expectations about the future. 
Figure 2 also depicts feedback loops from Oval 5 (Comprehension) to each of the 
preceding ovals and to the three lenses. (Not shown, but also included in the model are 
feedback loops from Oval 6 (Projection) to the five preceding ovals and to the three 
lenses.) 



Figure 2. Distortions in the lenses lead to inaccurate perceptions, comprehensions, and 
projections. Feedback loops in the model represent decisions and updates to the lenses. 

The following example illustrates the data and information flow through the model and 
the role of the feedback loops. There is an enemy unit moving through the battlespace 
(Oval 1) but it has not yet been detected. At some point, sensors detect the motion of the 
enemy (Oval 2). Data from the sensors are sent to the friendly unit and appear on the 
decision maker’s workstation (Oval 3). If the workstation is configured properly and the 
decision maker is attending to the workstation (based on contents of Lens A), the data 
may be perceived (Oval 4). The decision maker may determine (Oval 5) there is an 
enemy of unknown size and strength on the battlefield based on the local context and his 
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or her experience (as well as other contents of the Lens B). Given that comprehension 
and knowledge of enemy doctrine (as well as other contents of Lens C), the decision 
maker may expect or project (Oval 6) the enemy to be of a certain size and move in a 
particular direction. Having made that projection, the decision maker may elect to 
reposition an unmanned aerial vehicle (UAV) to more closely monitor the enemy unit. 
This decision is represented by the feedback loop from Oval 6 to Oval 1. Once the UAV 
arrives on station, the sensors on board provide additional data, which flow from Oval 2 
to Ovals 3, 4, 5, and 6. The additional data from the UAV sensors will either confirm or 
correct the earlier comprehension and projection. 

Tactical Network Topology (TNT) Field Studies 

Each academic quarter, the Naval Postgraduate School (NPS) conducts a week-long field 
exercise at Camp Roberts, a California Army National Guard installation just north of 
Paso Robles, CA. The exercises are sponsored by Special Operations Command 
(SOCOM) and are referred to as Tactical Network Topology (TNT) experiments after the 
communication infrastructure that supports the activity. The TNT exercises provide 
excellent opportunities for NPS faculty and students (and selected organizations external 
to NPS) to test novel hardware or software applications in a field setting. As such, the 
TNTs are often a collection of demonstrations rather than scientific experiments. 
Nevertheless, the field setting, coupled with realistic scenarios, provides an attractive 
alternative to strictly controlled laboratory or computer-based simulation activities. 

TNT 07-1 was conducted from October 27 - November 3, 2006. Three days were 
dedicated to comparing two methods for conducting searches with unmanned aerial 
vehicles (UAVs). NPS researchers developed an Aerial Search Optimization Model 
(ASOM). Their model identifies the optimal search patterns based on the number and 
type of UAVs available, and also considers logistical, operational, terrain, and 
intelligence inputs. Performance on a detection task while using ASOM was compared 
with performance on a manual (non-ASOM) search strategy based on doctrinal principles 
for employment of UAVs. Figures 3a, 3b, and 3c are photos of the three different UAVs 
used in this field experiment. They are relatively small and have a limited payload (i.e., a 
video camera with tilt, pan, and zoom capabilities). They were chosen because they are 
representative of UAVs currently being used by company- and platoon-sized units in 
coalition operations around the world. 



Figures 3a - 3c. Photos of the UAVs flown in support of TNT 07-1. Figure 3a (left) is a 
Buster UAV; Figure 3b (center) is a Raven UAV; Figure 3c (right) is a Tern UAV. 
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A total of 12 runs were completed, six ASOM and six non-ASOM. Each run was 48 
minutes in length. The scenarios for all 12 runs were similar. The UAVs were employed 
to detect and identify four enemy vehicles moving through the Camp Roberts terrain. 
The vehicles had entered the area through a checkpoint but shortly after being processed, 
a routine database check had revealed that the vehicles contained suspected enemy 
personnel who were wanted for questioning. The UAVs were launched in an attempt to 
locate the vehicles. There were from one to three UAVs available to conduct the search. 
There were six checkpoints (two in the west, two in the south, and two in the east) that 
could be used. And, there were 20 possible destinations for the four enemy vehicles. 
Each run started after the enemy vehicles passed through the designated checkpoint. 
Eight minutes later, the UAVs were released to begin the search. The run concluded 40 
minutes later or after all enemy vehicles had been detected and identified, whichever 
happened sooner. The enemy vehicles were sport utility vehicles (SUVs) with marker 
panels affixed to their roofs to facilitate identification. When an enemy vehicle was 
positively identified, the driver of that vehicle was instructed (via radio) to pull off the 
road and open the vehicle’s doors. This procedure was implemented in order to minimize 
the number of multiple detections. 

Prior to the TNT, the researchers divided the Camp Roberts terrain into 17 ASOM 
segments (see Figure 4a). Prior to an ASOM run, the researchers were told the type and 
number of UAVs that were to be available and the entry point for the enemy vehicles. 
Their model then generated a search pattern for each of the UAVs in the form of a 
sequence of segments. 



Figures 4a and 4b. Map on the left shows the Camp Roberts terrain overlaid with the 17 
ASOM segments (4a); map on the right shows the three non-ASOM search areas (4b). 
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The non-ASOM searches were conducted by dividing the Camp Roberts terrain into three 
areas - north, central, and south (see Figure 4b). These manual searches were conducted 
at the discretion of experienced UAV operators and according to the following doctrinal 
principles. 

• Within the first 10 minutes of the event, bias the search toward the sub-area near 
the suspected enemy point of entry. 

• Within the next 20 minutes, bias the search of the sub-area towards the areas of 
known High Value Targets (HVTs). 

• During the last 10 minutes, bias the search of the sub-area towards the direction of 
potential threat exit. 

• Actual method/technique (road following, bow tie, linear, etc.) of search is at the 
discretion of the UAV operators and per their specific UAV SOPs. 


For each ASOM run there was a corresponding non-ASOM run. These paired runs were 
identical with respect to the UAVs available for the search, the point of entry of the 
suspected enemy vehicles, and the destination of the vehicles. The order of the runs was 
randomized. Key players in these runs included the Tactical Operations Center (TOC) 
commander, the air boss, the two UAV video feed observers in the TOC, the UAV 
operators in the field, and the red team commander. The TNT mesh network carried the 
live video feed from the UAVs to the TOC. The role of the TOC commander was 
supervisory in nature. The air boss communicated directly with the three UAV ground 
control units (GCUs) and directed the administrative and operational activities of the 
UAVs, including search patterns. The UAV operators at the GCUs and the video feed 
observers in the TOC were responsible for detecting and identifying the suspected enemy 
vehicles. The red team commander directed the activities of the four enemy ground 
vehicles. 



Figures 5a and 5b. Photo on the left (5a) shows the work area of the air boss; photo on the 
right (5b) shows the work area of the video feed observers in the TNT tactical operations 
center. 

Employing the Dynamic Model of Situated Cognition in a Field Setting 

Prior to TNT 07-1, the authors of this paper used the Dynamic Model of Situated 
Cognition (DMSC) to analyze data generated by simulated command and control 
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exercises. In these studies, data generated by the simulation (and stored in memory) and 
the voice transcripts of the participants were used afterward to populate the ovals of the 
model. Since the TNT exercises are field studies, there is no single source from which to 
extract the data. Therefore, the authors of this paper had to coordinate with TNT exercise 
planners beforehand to identify various sources of data that could be collected and used 
to populate the ovals of the DMSC. 

All UAVs and enemy ground vehicles were equipped with global positioning system 
(GPS) tracking devices. Positions were recorded and stored once every second. (In 
addition to knowing the positions of the UAVs, the technology made it possible to 
reconstruct the footprint of each UAV camera as it searched the terrain below.) In the 
TOC, video feed observers typed their comments and observations into a time-stamped 
digital log. Activities in the TOC were also recorded with a digital video camera. Digital 
voice recorders were placed on the TOC commander, the air boss, the TOC video feed 
observers, and the red team commander. At the conclusion of TNT 07-1, GPS data were 
collected and analyzed, and a portion of the more than 50 hours of voice recordings were 
transcribed. 

At the conclusion of the TNT exercise, the authors used the GPS data and digital 
recording transcriptions to reconstruct the flow of data through the Dynamic Model of 
Situated Cognition. The purpose of this analysis was to determine what data were 
present in the technological portion of the system (i.e., the left side of the model) and 
what data were actually detected and processed by the humans in the system (i.e., the 
right side of the model). The results will be used to improve the TNT infrastructure, 
displays, and configuration of the TOC for future exercises. 




Figure 6a and 6b. Graph on the left (6a) depicts the hypothetical flow of data through the 
DMSC in an optimal way; graph on the right (6b) depicts the flow in a dysfunctional 
way. 

Figure 6a depicts the hypothetical profile of data flowing through the DMSC. As stated 
earlier, Oval 1 includes all data (i.e., 100%) in the environment. The column for Oval 2 
indicates that less than 75% of the data in Oval 1 are detected. Approximately 40% of 
the data proceed to Oval 3 (C2 workstations) and less than 30% make it to Oval 4 
(perception). The column in Oval 5 (comprehension) has the potential to be larger than 
the preceding two columns because decision makers may combine what they have 
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perceived with their experience and their understanding of both the situation and 
doctrine. The result is that their mental model of the situation may be a closer 
approximation of what is in Oval 1 than would be possible without the contents of the 
lens. The same process occurs from Oval 5 to Oval 6 (projection). An accurate 
comprehension coupled with an accurate lens will facilitate a projection that closely 
approximates the contents of Oval 1. 

Figure 6b depicts another possible flow through the model. In this scenario, erroneous 
data enter at Oval 2. The erroneous data are shown as columns that extend down from 
the X axis. The data may be erroneous because the algorithms in the sensors are flawed, 
the sensors lack specificity, or the enemy has successfully spoofed the sensor network. 
These erroneous data flow through the model to Oval 3 (C2 workstation) and to Oval 4 
(perception). The column in Oval 5 has a larger negative portion than previous columns 
because incorrect knowledge of doctrine or invalid experiences can distort the lens and 
lead to a comprehension of the situation that is only partially true. This inaccurate 
understanding will be the basis for a decision maker’s projection (Oval 6). As a result, 
the column in Oval 6 may have an even larger portion of the column that extends below 
the X axis. It also possible that “well-focused” lenses will reduce the erroneous data or 
that “poorly-focused” lenses would result in a more distorted projection at Oval 6. The 
situations illustrated in Figures 6a and 6b are just two of many permutations showing the 
quantity of veridical data and erroneous data at each oval. 

The analysis performed on the TNT data permitted the authors to construct a graph 
similar to those in Figure 6 that showed the extent to which the data flowed accurately 
through the DMSC. The authors reviewed the following sources of data in conducting 
their analysis: 

• voice transcripts, 

• video feed observer log, 

• data compiled from the GPS tracking devices on the UAVs and enemy ground 
vehicles, and 

• reports generated by a commercial software program that used the GPS tracking 
data to construct a simulation of the runs. 

These data were reviewed initially to identify hits (correct identification of enemy 
vehicles), misses (failure to detect enemy vehicles when they were present), false alarms 
(identifying an entity as an enemy vehicle when none was present), and correct rejections 
(classifying an entity as a non-target when it is not an enemy vehicle). Four of the runs 
were analyzed in detail. The analysis of one of those runs is presented here. 


Analysis of TNT Data 

Figure 7 shows the data from an ASOM run. This particular run employed three different 
UAVs (Raven, Buster, and Tern). The three columns for each UAV (Techno, GCU, and 
TOC) indicate which agents in the system detected an entity. The Techno columns 
identify technological detections. The data collected from the GPS trackers and analyzed 
after the run indicate that a UAV and an enemy vehicle were in close proximity to one 
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another. The GCU columns are detections by the operators at the UAV’s ground control 
unit. The TOC columns indicate detections by the TOC video feed observers. Detections 
were classified either as hits (yellow shading), misses (blue shading), false alarms (pink 
shading), or correct rejections (green shading). The four enemy vehicles were numbered 
R1 - R4. 
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Figure 7. Data from an ASOM Run that included three UAVs (Raven, Buster, and Tern) 


Start times for the detections were obtained from the GPS tracking data for technological 
hits and from the voice transcripts for the GCU and the TOC hits. The voice transcripts 
did not provide data with respect to the duration of the detection. Therefore, GCU and 
TOC detections do not have ‘Time End’ or ‘Total’ entries. The ‘Total’ column indicates 
the length of time (in seconds) the enemy vehicles were within a UAV camera’s 
‘footprint.’ Although the GPS tracking data were recorded once per second, one database 
used by the authors listed positions only every five seconds. Four other codes are present 
in the data: 

• Vt - a visual sighting of a suspected enemy vehicle by a video feed observer in 
the TOC; 

• Vg - a visual sighting of a suspected enemy vehicle by a GCU operator; 

• Tt - a confirmed sighting of an enemy vehicle by a video feed observer in the 
TOC; 
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• Tg - a confirmed sighting of an enemy vehicle by a GCU operator. 

Rows that include ‘Time Start’, ‘Time End’, and ‘Total’ entries were taken from reports 
generated by a commercial software program after the experiment. This software used 
every data point recorded (i.e., once per second) and then interpolated the data to provide 
exposure times for an enemy vehicle down to the millisecond. 

The authors used the technological data (i.e., the data from the GPS tracking devices) as 
the basis for the analysis. That is, if the technological database indicated that a UAV and 
its camera were in a position to detect an enemy vehicle, it was considered a 
technological hit. The voice transcripts and the TOC video feed observer logs were then 
examined to determine whether the technological hit was subsequently detected by a 
human either at the GCU or the TOC. For example, Row 8 in Figure 7 indicates that 
enemy vehicle R2 was detected by the Raven UAV for 4.384 seconds from 9:39:57 to 
9:38:01 but it was not detected by anyone at the GCU or in the TOC. Row 30 indicates 
that there was a confirmed sighting by the TOC’s Tern video feed observer but the GPS 
database did not show a technological hit. There also was no corresponding detection by 
the GCU. Therefore, the sighting by the TOC video feed observer was considered a false 
alarm. 

In this run, there were 22 technological hits, 16 by Raven, 6 by Buster, and 0 by Tern. 
There appear to be six detections by humans (see rows 3-4, 10-11, 14-16, 23, 30, 34 in 
Figure 7), 5 by Tern and 1 by Raven. Two of these detections were confirmed sightings 
(rows 14 and 30). However, since the six detections by humans were not preceded with 
technological hits (i.e., the GPS tracking database did not indicate that a UAV and an 
enemy vehicle were in close proximity), the human detections had to be classified as 
false alarms. Figure 8 depicts the flow of data through the DMSC for this run. This 
figure illustrates two possible explanations for the data in Figure 7. 




Figure 8a and 8b. Two possible flows through the DMSC. 

In Figure 8a, Oval 1 depicts the number of times in which the paths of the UAVs and the 
enemy vehicles actually intersected. Oval 2 contains some number of actual detections 
(hits) and some number of erroneous detections (false alarms). The data indicate that 
none of the six human detections coincided with the 22 technological hits. Therefore, the 
six total detections and the two confirmed human detections were classified as erroneous 
(false alarms). Since the authors were present in the TOC and were able to detect at least 
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some of the enemy vehicles on the live video feed and independently confirm the 
locations of these vehicles with the red team commander, it seems clear that not all of the 
human detections were erroneous. 

Figure 8b suggests an alternative flow of data through the model. Given the same 
number of times (approximately 10) the paths of the UAVs and the enemy vehicles 
intersected (Oval 1), a flawed algorithm in the technological system failed to detect any 
of the actual intersections but instead generated 22 erroneous technological hits. The 
UAV video feeds actually may have recorded six of the intersections (Oval 2), 
transmitted them to the GCUs and the TOC where all six were displayed (Oval 3) and 
perceived (Oval 4) and two were confirmed as enemy vehicles (Oval 5). 

Discussion 

Technology can be seductive. Data generated by sophisticated software programs and 
elegantly displayed on large flat panel screens give the appearance of precision and 
accuracy. However, on occasion, the data may be neither precise nor accurate. Such was 
the case with the software used to evaluate the ASOM data in this TNT exercise. The 
authors noticed the problems when they attempted to trace the flow of data through the 
DMSC. First, there was a false sense of precision. The GPS tracking data were accurate 
down to the second. However, the software used to generate the reports of the UAV and 
enemy vehicle intersections showed a level of precision down to the millisecond. 
Apparently, the software interpolates the data it is given and generates reports and 
representations based on those interpolations. Intersections of as little as 17 milliseconds 
- imperceptible to a human observer - were reported. 

Second, the algorithm used by the software was not properly constrained. In the 
scenario, all four enemy vehicles entered the battlespace through the same checkpoint at 
9:21 AM. The vehicles drove to different destinations for eight minutes before the UAVs 
began their searches. At 9:40 AM, 19 minutes into the run, the database indicated that 
the Raven detected all four enemy vehicles within eight seconds (see Figure 7, rows 17 - 
20). The authors did not believe this was possible, given that not all the vehicles were 
within the Raven’s sensor footprint at that time. Subsequent discussions with the 
commercial software developers revealed that their product determined the ‘detectability’ 
of an enemy vehicle by reconciling the location of the UAV and the angle of its camera 
with the location of the enemy vehicle. Distance from the UAV to the enemy vehicle is 
not considered. A UAV could be 50 miles away - well beyond the ability of a human to 
detect a vehicle with a low quality video camera - and the software would still register it 
as a hit. Had the authors not attempted to trace the flow of data through the DMSC, this 
software anomaly may not have been uncovered. Based on the authors’ findings the 
software developers constrained their algorithm to approximate the abilities of the human 
visual system. 

Third, there was a lag in the transmission of the video from the UAVs to the TOC. Video 
data were transmitted via one network while the telemetry data that provided exact 
location of the UAVs was sent on another network. The telemetry data was timely, but 
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the video feed varied from being near real time to lagging by as much as 15 minutes. 

This variable lag made it difficult to reconcile the technological data with the voice 
transcripts and the observer logs. 

Conclusion 

The Dynamic Model of Situated Cognition has proven to be useful for analyzing data 
from computer-based simulations and for retrospective analysis of accidents. This paper 
explored the utility of using the model in field studies. Our conclusion is that the model 
is a viable framework for analyzing data from field studies such as the TNT exercises and 
for identifying anomalies in command and control systems. It readily identifies 
breakdowns as data flow through technological systems to human agents. A significant 
challenge to using the model in field studies is determining how to collect data efficiently 
in order to populate each oval of the model. Audio and video tapes often require 
transcription or detailed coding. GPS tracking systems generate enormous databases that 
must be cleaned before they can be analyzed. And, of course, the ability to collect data 
on red team activities is essential for determining the extent to which the friendly forces 
understand the battlespace. The authors plan to continue using the Dynamic Model of 
Situated Cognition in field studies, to refine data collection methods for the model, and to 
use the model to influence the design of command and control systems. Such work is 
imperative if all of the tenets of Network Centric Warfare are to be fully realized. 
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Tenets of NCW 

- A robustly networked force improves information sharing 

- Information sharing and collaboration enhances the 
quality of information and shared situational awareness 

- Shared situational awareness enables collaboration and 
self synchronization, and enhances sustainability and 
speed of command 

- These in turn dramatically increase mission effectiveness 



(from Alberts & Hayes, 2003; Garstka & Pattillo, 2003) 
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The Enticement of Technology 





• Focusing on technological solutions with little 
consideration for the capabilities and limitations of the 
warfighters is imprudent. 

• Yet, this is often the case. 

• Result is that these novel technologies will provide 
capabilities not needed by warfighters or they function in 
ways that are not compatible with warfighters. 

• These solutions will quickly fall into disuse or will distract 
warfighters from the tasks truly important for mission 
accomplishment. 
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The Vacuous Contributions of Psychology 





• Equally unacceptable is the narrow focus of some 
psychologists on cognitive processes of the humans without 
considering the military context or the technologies with which 
the warfighters must interact. 

• While these researchers may gain valuable insights into 
human cognition, their findings by themselves may have limited 
applicability to warfighting. 

• These findings must be shared with technologists who can 
then incorporate them into the design of new systems. 

• Thus, cooperation and collaboration between technologists 
and psychologists or human performance experts are critical to 
the success of NCW. 
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Feedback Loops in the Model 
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Potential Sources of Error 




Flow of data from Oval 1 to Ovals 2 & 3 


Accurate 


Inaccurate 


Missed 


Sensors: ^ 

• Insufficient quantity 

• Malfunction 

• Lack of sensitivity 

• Lack of specificity 


Misidentified 

\ 

Flawed algorithm 
Spoofing 


Other Sources of Error 

• Data elements ‘drop out’ at various points. 

• Erroneous data are inserted or existing data are 
morphed. 

• Misshaped lenses skew perceptions, 
comprehensions, and projections. 

• Feedback loops are erroneous or absent. 
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Process Tracing Methodologies 





“ The goal in these methods is to map out how the incident 
unfolded including available cues, those cues actually noted by 
participants, and participants’ interpretation in both the immediate and 
in the larger institutional and professional contexts. This is called 
process tracing or protocol analysis method because it focuses on 
how a given outcome came about.” 

“The specific techniques within this family are all oriented towards 
externalizing internal processes or producing external signs that 
support inferences about internal workings .” 

“Process-tracing techniques primarily use data from verbal reports or 
from records of problem-solver behavior to build protocols that 
describe the sequence of information flow and knowledge 
activation .” 

(Woods, 1993) 
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Depiction of Data Flow through 
Dynamic Model of Situated Cognition 





Hypothetical flow of data through Hypothetical flow of data through 

the DMSC in an optimal way the DMSC in a dysfunctional way 
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Tactical Network Topology (TNT) 

Events 



• NPS conducts a week-long field exercise at Camp Roberts, CA each 
academic quarter. 

• Exercises sponsored by Special Operations Command (SOCOM). 

• Referred to as Tactical Network Topology (TNT) experiments after the 
communication infrastructure that supports the activity. 

• An excellent opportunity for faculty and students (and selected 
organizations external to NPS) to test novel hardware or software 
applications in a field setting. 

• The field setting, coupled with realistic scenarios, provides an attractive 
alternative to the laboratory or computer-based simulation activities. 
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UAVs Employed at TNT Event 
Camp Roberts, CA 
October 2006 




Buster Raven 
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Description of TNT 07-1 





• TNT 07-1 conducted 27 Oct - 3 Nov 2006. 

• Three days dedicated to comparing two methods for conducting 
searches with UAVs. 

- ASOM 

- Manual (non-ASOM) 

• 12 runs were completed - 6 ASOM and 6 non-ASOM. 

• UAVs were used to detect and identify 4 enemy vehicles 

• Each run was 48 minutes in length. 

• For each run, all enemy vehicles entered Camp Roberts thru 1 of 6 
checkpoints and proceeded to any of 20 locations. 

• ASOM and non-ASOM runs were paired (e.g. enemy start points and 
destinations for ASOM Run 1 were identical to non-ASOM Run 5) 
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Camp Roberts Terrain Map 






17 ASOM Segments 


Three Non-ASOM Search Areas 
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TNT Tactical Operation Center 
Camp Roberts, CA 





Work Area of Air Boss 


Work Area of Video Feed Observers 
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Data Collected at TNT 07-1 





• All UAVs and enemy ground vehicles were equipped with 
global positioning system (GPS) tracking devices. 

• Positions recorded and stored once per second. 

• Technology made it possible to reconstruct the footprint of 
each UAV camera. 

• Video feed observers (in TOC) typed their comments and 
observations into a time-stamped digital log. 

• Activities in the TOC were recorded with a digital video 
camera. 

• Digital voice recorders placed on TOC CDR, Air Boss, video 
feed observers, and the Red Team CDR. 
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Summary of Data for an ASOM Run 





• 22 technological hits (Raven - 16; Buster - 6; Tern - 0) 

• Only 6 detections by humans (Tern - 5; Raven -1). 

• However, since the 6 detections by humans were not 
preceded with technological hits (i.e., the GPS tracking 
database did not indicate a UAV and an enemy vehicle were 
in close proximity), the human detections had to be 
classified as false alarms. 
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Data from an ASOM Run with Three UAVs 
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Two Possible Flows Through the 
Dynamic Model of Situated Cognition 
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Technology Creates Challenges to Data 
Collection in Field Settings 



Technology can be seductive. Problems include: 

• Difficulties in determining ground truth. 

• Algorithms may be wrong but presumed correct. 

• Undetected lags in data transmission. 

• Elegantly displayed data give the appearance of 
precision and accuracy. 

• Decisions made based on bad data result in errant 
feedback loops which propagate errors through the 
system. 
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Lessons Learned at TNT 07-1 





• Capturing ground truth is critical. 

• Technological systems may not be trustworthy. 

• Data must be collected at all critical nodes and in formats 
that facilitate rapid analysis and synthesis. 

• Early involvement in the exercise planning increases the 
likelihood that the event will result in an empirically sound 
field study. 
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